1. Field of the Invention
The invention, in general, relates to a method of taking three-dimensional measurements of an object surface and, more particularly, to a method of projecting patterns onto an object surface and of defining the object coordinates by analyzing the similarity of the grey scale gradation in images of the object surface taken by at least two matrix cameras.
2. The Prior Art
Methods heretofore known of optically measuring three-dimensional objects at close range are, for instance, photogrammetric methods or stripe-projection methods. Whereas in the stripe-projection method striped patterns which are geometrically as exact as possible are projected onto the object to be measured, the patterns used in photogrammetric methods may be more or less arbitrary.
The projector for projecting patterns onto an object surface will hereinafter sometimes be referred to a pattern projector. The device for generating an image of the object surface and the sensor for recording the intensity distribution, pixel by pixel, in the image will hereafter sometimes be referred to as matrix camera. More specifically, in the present context matrix cameras are understood to include CCD cameras and other apparatus suitable for recording an intensity distribution in a pixel by pixel manner. A point in the sub-pixel range is to understood as a position on the (camera) sensor which is positioned between pixel centers. Since it is generally assumed that the measured grey scale values are valid only for the pixel centers, the grey scale value for a point in the sub-pixel range is interpolated from surrounding pixels. For instance, bilinear interpolation is one method by which the grey value may be calculated on the basis of the surrounding four pixels. The position on the sensor of the second matrix camera, hereinafter sometimes referred to as the corresponding point, may be determined relative to a selected pixel of the sensor of a first camera by projecting the same portion of the surface of the body to be measured on the selected pixel and on the corresponding point.
In the simplest case, the measurement structure for practicing a stripe-projection method consists of a pattern projector for projecting a striped pattern, a matrix camera of the kind disclosed, for instance, by German patent specification 3,938,714, and the object to be measured. The form or shape of the stripes projected onto the surface and affected in a determined manner by the object, is recorded by the sensor and may be evaluated by a computer connected therewith.
The so-called phase shift method is a special method which ensures relatively high accuracy of the measured 3D-coordinates. Essential aspects of the evaluation of the intensity distributions recorded by the matrix cameras relate to defining the absolute order of the individual stripes as well as the phase position of each image point. For the latter, Nxe2x89xa73 individual exposures of the object surface are sequentially taken, with the periodic striped pattern projected onto the object surface being shifted by an nth part of the stripe period between each exposure. Accordingly, a set of N individual images is obtained for each image point.
However, measuring coordinates in this manner suffers from a disadvantage inherent in the phase-shift method, i.e. the need for calibrating the pattern projector. This is because of the principle which requires defining the 3D coordinates by means of the matrix camera and the projector, i.e. by triangulation generally. Accordingly, the projector has to perform a function which is usually assumed by the camera. In general it is, however, more difficult or complex to calibrate a projector than it is to calibrate a camera. Moreover, the distortion parameters are generally larger and cannot be as easily determined as in the case of a camera. This, in turn, may result in a greater uncertainty in the calculation of the spatial coordinates.
A further problem arises from the fact that oftentimes the grey scale gradation of the stripes is not sinusoidal in a way which may be considered to be ideal. Any such deviations may have a direct distorting effect on the results of the measurement and lead to a lower accuracy of the measurements.
In photogrammetric methods, the calculation of the coordinates of the surface is based upon images of the object surface taken by at least two cameras. One advantage of photogrammetry is its high measuring accuracy. Yet where only one pair of images is used, measurements may become critical at or near strong curvatures or other irregularities in the object surface. The reason for this is that the position of corresponding image points is usually determined by analyzing similarities in the grey scale gradation of two camera images within a defined measuring area of but a few, e.g. 15xc3x9715, pixels. Usually, it is the maximum of the correlation function or the minimum of the sum of the square of errors of these two grey scale gradations which is used as a measure of the similarities.
Accordingly, the accuracy of measurements obtained by this method is strongly dependent upon the size of the measured surface and critical surfaces, i.e. strongly curved or highly irregular surfaces generally cause systematic measurement errors. While this disadvantage could be lowered by reducing the size of the measurement surface, it would result in a reduced number of pixels for analyzing the grey scale gradation and, therefore, in a less exact position of the corresponding points determined by an analysis of the similarities. Accordingly, seeking to solve the problem in this manner would seem at best to be of limited usefulness.
An alternative measuring method has been disclosed by German Patent specification DE 196 23 172 in which the correlation surface consists of but one pixel. The method can be realized by initially projecting a sequence of n (for instance 12) different patterns onto an object surface and by recording a corresponding number of image pairs. Accordingly, a sequence of n grey values becomes available of a pixel selected by one camera. The correlation coefficient between this sequence of grey values and the sequence of grey values of a point of the other camera is drawn upon to determine the corresponding point. The algorithm is based upon a search for the correlation maximum along the epipolar line. The method suffers from the drawback that it is not possible within the coordinate system of the object to measure either equidistantly or along predetermined directional vectors within the object coordinate system.
Accordingly, it is an object of the invention to provide for equidistant measurements in an object coordinate system in a method of three-dimensionally measuring object surfaces in which patterns are projected onto the object surface and of determining the object coordinates by analyzing the similarity of the grey scale gradation in images of the object surface recorded by at least two cameras.
Another object is to provide for measuring along directional vectors in an object coordinate system in the context of a method of the kind referred to supra.
In the accomplishment of these and other objects, the invention in a currently preferred embodiment thereof provides for a method of three-dimensionally measuring an object surface in which predetermined patterns are projected onto the object surface and spatial coordinates of measuring points on the object surface are determined by analysis of the similarity of the grey scale gradation in images of the object surface recorded by at least two matrix cameras, and including the steps of:
a) each matrix camera generates a set of N sequential images whose pattern is changed by predetermined values between individual exposures,
b) a first approximation point is determined for calculating a first measuring point,
c) coordinates in the sensor surfaces of the two matrix cameras are correlated by mathematical projection to the approximation point or with a number of points adjacent to the approximation point,
d) the grey values of the N images of each matrix camera at the positions of the projected points are determined by subpixel interpolation and a pair of grey values is derived therefrom,
e) proceeding from the determined approximation point of step b), mxe2x89xa71 further approximation points are determined for calculating the first measuring point and for correlating coordinates in the sensor surfaces of the two matrix cameras with these m approximation points or the number of points adjacent to the approximation point,
f) the grey values in the N images of each matrix camera at the positions of the points projected in step e) are determined by subpixel interpolation and m pairs of grey scale values are derived therefrom,
g) the measures of similarity of the grey scale values associated with each one of the m+1 pairs are determined, and
h) the approximation point or a number of points adjacent to it for which the pair of grey scale values determined by steps e) and f) is of maximum similarity, is derived as the measuring point on the surface of the object measured.
The characteristic function of the measuring method in accordance with the invention which may be realized by a structure consisting essentially of at least two matrix cameras and a pattern projector as well as an object to be measured, is substantially as follows:
A pattern is sequentially projected N times onto the object surface to be measured at predetermined intervals such that temporarily N different intensity distributions are generated on the object surface. Accordingly, a sequence of N temporary intensity values may be correlated to each point of the surface area to be measured. For practical purposes, the structure of the pattern and the manner of the projection must satisfy the condition that in the sequence of N intensity values gradients as high as possible occur at each point of the object surface to be measured. Each one of the temporary intensity gradations generated in this manner is recorded by at least kxe2x89xa72 matrix cameras. The grey values in the N images thus generated will then be determined for each camera in a pixel by pixel manner.
The first approximation point is thereafter determined so that a first measuring point on the object surface may be calculated. Coordinates on the sensor surfaces of the two matrix cameras are thereafter correlated by mathematical projection to this approximation point or to a number of points adjacent to the approximation point.
The grey values in the N images of each matrix camera at the positions of the projected points are determined by subpixel interpolation, and a pair of grey scale values is derived therefrom.
At least one further approximation point is determined near the first approximation point. Coordinates in the sensor surfaces of the two matrix cameras are in turn correlated by mathematical projection to these approximation points or numbers of points adjacent to the approximation point. Thereafter, the grey values in the N images of each of the two matrix cameras at the positions of the projected points are again determined by subpixel interpolation, and at least one further pair of grey scales is derived therefrom. In a next step the measures of similarity pertaining to each of the above mentioned pairs of grey values are determined. Thereafter, the measuring point on the surface of the measured object is determined by finding the approximation point or a number points adjacent thereto to which one of the pairs of grey scale values determined by the above steps bears a maximum similarity.
Because of its local resolution and the low degree of uncertainty of measurements made possible by it, the measuring method in accordance with the invention is particularly suited for measuring vehicle bodies or parts thereof.
In contrast to known measuring methods, measurements of high resolution may be obtained by practicing the method of the invention, and when determining a measuring point value it is possible in a simple manner to ensure that the measuring points are provided equidistantly in the object coordinate system. This is deemed to be a significant advantage over the method disclosed by German patent specification DE 196 23 172.
It is also possible to shift the approximation points along predetermined directional vectors in the object coordinate system in order to obtain a measuring point at the point of penetration of the vector through the object surface.
Other objects will in part be obvious and will in part appear hereinafter.
The rate at which the measurements can be taken can be enhanced by utilizing at least two similarity measures of grey scale gradations for determining a new approximation point. This may be accomplished, for instance, by a gradient slope method or by Newton""s method. Such method would lead to the new approximation point being positioned substantially closer to the real surface of the measurement object than the first approximation points. This, in turn, simplifies the determination of the measuring point.
In an embodiment considered to be particularly advantageous, the directional vector along which the approximation points are shifted is positioned parallel to the axis of the object coordinate system related to the height coordinates of the measuring points. The measuring point may thus be determined equidistantly.
In a further advantageous embodiment of the invention the coordinates of new approximation points are determined by considering the measures of similarities correlated to a predetermined number of approximation points.
Further advantageous embodiments relate to the design of the projected patterns. Striped patterns have been found to be effective the longitudinal direction of the stripes of which imaged on the sensor planes of the matrix cameras extends substantially normal to the epipolar lines. They yield favorable grey scale gradients and, accordingly, a high information content along these lines. In turn, it has a favorable effect on the accuracy of determining the position of the corresponding points.
If possible, the grey scale values transverse of the direction of the lines should differ in the entire area to be measured. This would be the case, for instance, with a sinusoidal grey scale gradation. The necessary N different intensity distributions on the surface to be measured may be generated in different ways. In a first variant, for instance, the striped pattern may be shifted by a fraction of its period between individual images. In another variant, the different intensity distributions may be obtained by the projection of striped patterns of differing period or by a combination of both variants.
In yet another embodiment of the invention the projected patterns are of a stochastic or random structure. In that case, the sequentially different intensity distributions on the surface of the object are generated by translation and/or rotation of the projected pattern, or by changing the structure or outline of the pattern.